Learning device, learning method, inference device, and storage medium

ABSTRACT

A learning device includes: a data acquisition unit that acquires learning target data that is full-size data of a learning target; a data generation unit that divides the learning target data to generate multiple pieces of first divided data that is divided data of the learning target data, and adds, to each piece of the first divided data, first identification information for identifying a region of the first divided data in the learning target data; and a model generation unit that generates a learned model for determining an anomaly in the first divided data using first correspondence information that is a set of the first divided data and the first identification information corresponding to the first divided data.

FIELD

The present disclosure relates to a learning device, a learning method,and an inference device for learning a normal state of an inspectiontarget.

BACKGROUND

In manufacturing some products, inspection of the products may beperformed by machine learning. A learning device that performs machinelearning is configured to inspect manufactured products using amultilayered neural network, for example.

The anomaly detection device described in Patent Literature 1 receivesnormal image data for learning as an input, and performs machinelearning such that its output data becomes the same as the normal imagedata for learning. This anomaly detection device extracts featurecomponents from image data of an inspection target based on the learningresult, and determines the absence or presence of an anomaly in theinspection target based on a difference between the feature componentsand the image data of the inspection target.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent Application Laid-open No.2018-195119

SUMMARY Technical Problem

However, it takes a longer time to learn if learning data such as imagedata has a larger data size. A possible method to shorten the learningtime is to divide the learning data into pieces and perform machinelearning using the divisional pieces of learning data. Suppose that thismethod is applied to the anomaly detection device described in PatentLiterature 1. In this case, learning data for determining the absence orpresence of an anomaly is divided into a plurality of learning datapieces, but the position of each of the divisional learning data piecesis not managed at all. Therefore, if a learning data piece exhibiting anabnormal sate is the same as a learning data piece exhibiting a normalstate at a position different from a position of the abnormal learningdata piece, the anomaly detection device erroneously determines that theabnormal learning data piece corresponds to a learning data exhibiting anormal state, and cannot execute accurate machine learning.

The present disclosure has been made in view of the above circumstances,and an object thereof is to provide a learning device capable ofaccurately performing machine learning of the normal state of aninspection target in a shorter time.

Solution to Problem

In order to solve the above-mentioned problems and achieve the object,the present disclosure provides a learning device comprising: a dataacquisition unit to acquire learning target data that is full-size dataof a learning target; a data generation unit to divide the learningtarget data to generate multiple pieces of first divided data that isdivided data of the learning target data, and add, to each piece of thefirst divided data, first identification information for identifying aregion of the first divided data in the learning target data; and amodel generation unit to generate a learned model for determining ananomaly in the first divided data using first correspondence informationthat is a set or the first divided data and the first identificationinformation corresponding to the first divided data.

Advantageous Effects of Invention

The learning device according to the present disclosure has anadvantageous effect that it can accurately perform machine learning ofthe normal state of an inspection target in a shorter time.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a learning deviceaccording to a first embodiment.

FIG. 2 is a diagram illustrating a configuration of a neural networkthat is used by the learning device according to the first embodiment.

FIG. 3 is a flowchart illustrating a procedure for learning processingperformed by the learning device according to the first embodiment.

FIG. 4 is a diagram illustrating a configuration of an inference deviceaccording to the first embodiment.

FIG. 5 is a flowchart illustrating a procedure for inference processingperformed by the inference device according to the first embodiment.

FIG. 6 is a diagram illustrating a configuration of a learning devicehaving a function of the inference device described in FIG. 4 .

FIG. 7 is a diagram illustrating a hardware configuration of thelearning device illustrated in FIG. 6 .

FIG. 8 is a diagram for explaining normal data learning processing andinference processing, performed by the learning device illustrated inFIG. 6 .

FIG. 9 is a diagram for explaining a difference between an anomalyindication map generated by the inference device according to the firstembodiment using cutting position information and the anomaly indicationmap generated without using the cutting position information.

FIG. 10 is a diagram for explaining a cutting image set in divided datafor three channels of RGB, colors by a learning device according to asecond embodiment.

FIG. 11 is a diagram for explaining a cutting image set in divided datafor one channel of grayscale by the learning device according to thesecond embodiment.

FIG. 12 is a diagram for explaining a data array image by which adivided data cut region set in divided data for one channel of grayscaleby the learning device according to the second embodiment isidentifiable.

FIG. 13 is a table for explaining a difference between a calculationtime in the case where an inference device according to the secondembodiment divides a full-size image to execute machine learning and acalculation time in the case where machine learning is executed withoutdividing the full-size image.

FIG. 14 is a diagram for explaining processing in which an inferencedevice according to a third embodiment directly inputs the cuttingposition information on divided data to an intermediate layer of thelearned model.

DESCRIPTION OF EMBODIMENTS

Hereinafter, a learning device, a learning method, and an inferencedevice according to embodiments of the present disclosure will bedescribed in detail with reference to the drawings. The presentdisclosure is not necessarily limited by these embodiments.

First Embodiment

FIG. 1 is a diagram illustrating a configuration of a learning deviceaccording to the first embodiment. The learning device 10 is a computeradapted to divide learning target data that is a learning sample of aninspection target into pieces, and performs machine learning of theinspection target having a normal state using the divisional learningtarget data pieces. The learning device 10 generates a learned model bythe machine learning that is an example of artificial intelligence (AI),and an inference device (described later) uses the generated learnedmodel to infer the inspection target having a normal state, and comparesthe inspection target with the inspection target having, a normal state,thereby to determine whether or not the inspection target is abnormal.Hereinafter, description is given for a case where the learning targetand the inspection target are image data of a product, but the learningtarget and the inspection target may be data different from image data.

The learning device 10 may execute machine learning in any learningmethod. For example, the learning device 10 may execute machine learningthrough non-defective product learning using image data of non-defectiveproducts, or may execute machine learning using image data ofnon-defective products and defective products.

The learning device 10 divides the entire image data (hereinafterreferred to as entirety data) to generate multiple divisional image datapieces (hereinafter referred to as divided data or divided data pieces).An example of the entirety data is data indicating an image of the wholeof a product, that is, data representing a full-size image thereof, andan example of the divided data is data indicating a partial region of aproduct. The entirety data may be image data of an image of a productcaptured in any direction. In the first embodiment, description is givenfor a case where the entirety data is image data of an image of aproduct captured from the upper position.

The learning device 10 performs machine learning with associating adivided data piece with the position of the divided data piece in theentirety data. The learning device 10 includes a data acquisition unit11, a data generation unit 16, and a model generation unit 14. The datageneration unit 16 includes a data cutting unit 12 and a positioninformation adding unit 13.

The data acquisition unit 11 acquires entirety data. The dataacquisition unit 11 acquires the entirety data from an external devicesuch as an imaging device that captures an image of a product, forexample. The data acquisition unit 11 sends the acquired entirety datato the data generation unit 16.

The data cutting unit 12 divides the entirety data and cuts the divideddata piece off from the entirety data. Note that the divided data piecemay have an arbitrary size. For example, the entirety data is data of arectangular image one side of which has 1024 pixels. The divided datapiece is data of a rectangular image whose side has 64 pixels.

The position information adding unit 13 adds, to the divided data piece,cutting position information for identifying the region of the divideddata piece in the entirety data. Cutting position information isrepresented by, for example, the coordinates of two or more vertices ofthe divided data piece in the entirety data. Note that cutting positioninformation may be represented by the coordinates of one point in adivided data piece in the entirety data and a size.

The data generation unit 16 sends the cutting position information addedby the position information adding unit 13 and the divided data piecegenerated by the data cutting unit 12 to the model generation unit 14while associating the cutting position information with the divided datapiece.

The model generation unit 14 generates a learned model 31 usingcorrespondence information that is information in which the cuttingposition information and the divided data are associated with eachother. In the first embodiment, the correspondence information islearning data for learning image data having a normal state (hereinafterreferred to as normal data), that is, learning target data. In a casewhere the entirety data is divided into 100 pieces of divided data, themodel generation unit 14 generates the learned model 31 using thecorrespondence information for the 100 pieces.

The model generation unit 14 learns the normal data on the basis of thecorrespondence information (learning data) generated based on acombination of the cutting position information and the divided datapiece. That is, the model generation unit 14 generates the learned model31 used to infer normal data from the cutting position information andthe divided data piece. The model generation unit 14 generates onelearned model 31 for the entirety data. Here, the learning data is datain which the cutting position information and the divided data piece areassociated with each other.

The learned model 31 is configured to perform machine learning of imagedata of a product having a normal state using, for example, deeplearning in which neural networks are stacked in a multilayered manner.In response to input or the correspondence information, the learnedmodel 31 outputs image data having a normal state. In response to inputof one piece of correspondence information, the learned model 31 outputsone piece of normal data.

The image data of the normal data outputted by the learned model 31 isimage data of the same region as the divided data piece. In response toinput of the divided data piece that is learning data, the learned model31 performs machine learning such that the output data is image datarepresenting a normal state. The learned model 31 performs machinelearning of normal data, i.e., image data representing the producthaving a normal state, while extracting feature data characteristic ofthe learning data from the learning data. That is, the model generationunit 14 uses deep learning that extracts and learns a feature quantityas a learning algorithm to cause the neural network to learn whilecausing the network to extract features using a set of multiple piecesof learning target data that are normal learning samples as theinspection target. Consequently, the model generation unit 14 performs asort of unsupervised learning with no input of a feature and with noinput of any explicit teacher signal indicating that it is normal oranomaly, or the like and generates the learned model 31 that outputsnormal data appropriate as normal data having a normal feature of theinspection target.

In the case where the learned model 31 is a model that uses a neuralnetwork, the neural network is composed of an input layer consisting ofa plurality of neurons, an intermediate layer (hidden layer) consistingof a plurality of neurons, and an output layer consisting of a pluralityof neurons. The number of intermediate layers may be one, or two ormore.

FIG. 2 is a diagram illustrating a configuration of a neural networkthat is used by the learning device according to the first embodiment.For example, in the case where the neural network is a three-layeredneural network as illustrated in FIG. 2 , when a plurality of inputs isinputted to an input layer (X1 to X3), the values thereof are multipliedby weights W1 (w11 to w16) and inputted to an intermediate layer (Y1 toY2), and the results thereof are further multiplied by weights W2 (w21to w26) and outputted from an output layer (Z1 to Z3). The outputresults vary depending on the values of the weights W1 and W2.

The neural network learns normal data appropriate as normal data havingone or more normal features through deep learning based oncorrespondence information that is a set of divided data piece generatedby the data generation unit 16 and identification informationcorresponding to the divided data piece.

That is, the neural network performs learning based on adjustment of theweights W1 and W2 such that the correspondence information that is a setof the divided data piece and the identification informationcorresponding to the divided data piece is inputted to the input layer,a feature or features are extracted in the intermediate layer, andnormal data appropriate as normal data having the extracted features isoutputted from the output layer.

The model generation unit 14 executes the learning as lust described togenerate the learned model 31, and outputs the generated learned model31. A learned model storage unit 15 is a device configured to store thelearned model 31 outputted from the model generation unit 14. Thelearned model storage unit 15 may be placed inside the learning device10 or may be placed outside the learning device 10.

In the above-described case, a sort of unsupervised learning based ondeep learning is used as the learning algorithm used by the modelgeneration unit 14. However, publicly known algorithms may be used suchas other types of unsupervised learning, supervised learning,send-supervised learning, and reinforcement learning. In addition, inthe above-described case, a configuration of the neural network for deeplearning is used as the configuration of the learned model 31. However,a configuration based on a different learning algorithm may be used.

Next, learning processing by the learning device 10 will be describedwith reference to FIG. 3 . FIG. 3 is a flowchart illustrating aprocedure for learning processing performed by the learning deviceaccording to the first embodiment. The data acquisition unit 11 acquiresentirety data indicating an image of the whole of a product (step S10).The entirety data acquired here is used for the machine learning ofnormal data. The data generation unit 16 divides the entirety data andcuts the pieces of divided data from the entirety data (step S20).

The position information adding unit 13 adds, to a divided data piece,cutting position information for identifying the region of the divideddata piece in the entirety data (step S30). The data generation unit 16sends, to the model generation unit 14, correspondence information inwhich the cutting position information added by the position informationadding unit 13 and the divided data piece generated by the data cuttingunit 12 are associated with each other.

The model generation unit 14 executes the learning process of thelearned model 31 (step S40). Specifically, the model generation unit 14generates the learned model 31 using the correspondence information inwhich the cutting position information and the divided data piece areassociated with each other. In other words, the model generation unit 14learns normal data through what is called unsupervised learning, inaccordance with the learning data generated based on the combination ofthe cutting position information and the divided data piece, andgenerates the learned model 31. The learned model storage unit 15 storesthe learned model 31 generated by the model generation unit 14 (stepS50).

Next, an inference device that infers normal data using the learnedmodel 31 and determines whether an anomaly occurs in inference targetdata will be described. FIG. 4 is a diagram illustrating a configurationof an inference device according to the first embodiment. The inferencedevice 20 includes a data acquisition unit 21, a data generation unit26, and an inference unit 24. The data generation unit 26 includes adata cutting unit 22 and a position information adding unit 23.

The data acquisition unit 21 has the same function as the dataacquisition unit 11, and the data generation unit 26 has the samefunction as the data generation unit 16. Whereas the data acquisitionunit 11 acquires entirety data to be used as learning data, the dataacquisition unit 21 acquires entirety data to be used as inspectiondata. The entirety data to be used as inspection data is inferencetarget data. In addition, whereas the data generation unit 16 generatesthe correspondence information to be used as learning data, the datageneration unit 26 generates correspondence information to be used asinspection data. The data generation unit 26 sends the generatedcorrespondence information to the inference unit 24.

The inference unit 24 receives the correspondence information sent fromthe data generation unit 26. The inference unit 24 infers normal datafrom the correspondence information with use of the learned model 31stored in the learned model storage unit 15. That is, the inferencedevice 20 infers normal data appropriate as normal data having normalfeatures of the correspondence information, based on input of thecorrespondence information of the inspection data to the learned model31. At this time, the inference unit 24 infers normal data for eachdivided data piece on the basis of the divided data piece of theinspection data and the cutting position information that serves asidentification information.

The inference unit 24 compares the normal data that is the inferenceresult with the divided data piece in the correspondence informationreceived from the data generation unit 26, and identifies a portion inwhich a certain difference occurs between the normal data and thedivided data piece, as an abnormal portion. At this time, the inferenceunit 24 determines, for each divided data piece, the difference betweenthe normal data and the divided data.

The inference unit 24 generates an anomaly indication map 33 indicatingthe abnormal portion on the basis of the determination result for eachdivided data piece, and outputs the anomaly indication map 33 to anexternal device as an inspection result. The anomaly indication map 33is a map that shows the abnormal portion of the entire image data in adifferent color or in any other manner such that the abnormal portion isdistinguishable from other portions. The anomaly indication map 33 isobtained by overwriting the entirety data with the mapped abnormalportions. In a case where the external device to which the inferenceunit 24 outputs the anomaly indication map 33 is a display device suchas a display, the display device displays the anomaly indication map 33.Note that the external device to which the anomaly indication map 33 isoutputted may be an alarm device or the like.

In the above-described example, the inference unit 24 generates theanomaly indication map 33 indicating the abnormal portions. However, theinference unit 24 need not necessarily identify the abnormal portion,and may determine whether each divided data piece is abnormal or not,then determine that the entirety data of the inspection target isabnormal if any of the divided data pieces is abnormal, and output onlythe determination of whether there is an anomaly as the inspectionresult. In addition, the inference unit 24 may first combine the piecesof normal data associated with the respective pieces of divided data togenerate the normal data for a full-size image corresponding to theentirety data, and then compare the entirety data with the normal datacorresponding to the entirety data to thereby determine an anomaly.

In the first embodiment, description is given for an example in whichthe inference device 20 outputs the anomaly indication map 33 using thelearned model 31 obtained by the learning in the model generation unit14 of the learning device 10, but the inference device 20 may use thelearned model 31 obtained by learning in a device different from thelearning device 10. In the latter case, the inference device 20 mayacquire the learned model 31 from another external device or the likedifferent from the learning device 10, and output the anomaly indicationmap 33 based on this learned model 31.

Note that the learning device 10 and the inference device 20 may beconnected via a network, for example. In this case, the learned model 31generated by the learning device 10 is sent to the inference device 20via the network.

Next, inference processing performed by the inference device 20 will bedescribed with reference to FIG. 5 . FIG. 5 is a flowchart illustratinga procedure for inference processing performed by the inference deviceaccording to the first embodiment. The data acquisition unit 21 acquiresentirety data indicating an image of the whole of a product (step S110).The entirety data acquired in this step is used for the inferenceprocessing of normal data. The data generation unit 26 divides theentirety data and cuts the pieces of divided data from the entirety data(step S120).

The position information adding unit 23 adds, to each divided datapiece, cutting position information for identifying the region of thedivided data piece in the entirety data (step S130). The data generationunit 26 sends, to the inference unit 24, correspondence information inwhich the cutting position information added by the position informationadding unit 23 and the divided data piece generated by the data cuttingunit 22 are associated with each other.

The inference unit 24 receives the correspondence information sent fromthe data generation unit 26. The inference unit 24 infers normal datafrom the correspondence information using the learned model 31 stored inthe learned model storage unit 15 (step S140). At this time, theinference unit 24 infers the normal data for each piece of divided data.The learned model 31 used by the inference unit 24 outputs the normaldata in response to the input of the correspondence information in whichthe cutting position information and the divided data are associatedwith each other.

The inference unit 24 compares the normal data that is the inferenceresult with the divided data piece in the correspondence informationreceived from the data generation unit 26 (step S150), and determinesthe difference between the normal data and the divided data piece basedon the comparison result. At this time, the inference unit 24determines, for each piece of divided data, the difference between thenormal data and the divided data piece. The inference unit 24 generatesthe anomaly indication map 33 based on the difference between the normaldata and the divided data piece (step S160). The inference unit 24outputs the generated anomaly indication map 33 to an external device.

Note that the inference device 20 may execute learning for normal datawhile inferring the normal data using the learned model 31. In thiscase, the learned model 31 is updated to a new learned model 31 as theresult of the learning performed by the inference device 20.

In addition, the learning and inference of the normal data may beexecuted by a learning device in which the learning device 10 and theinference device 20 are combined. FIG. 6 is a diagram illustrating aconfiguration of a learning device having a function of the inferencedevice described in FIG. 4 . The learning device 10A having a functionof the inference device 20 includes a data acquisition unit 11A, a datageneration unit 16A, the model generation unit 14, a learned modelstorage unit 15A, and the inference unit 24. The data generation unit16A includes a data cutting unit 12A and a position information addingunit 13A. The data acquisition unit 11A serves as both the dataacquisition unit 11 and the data acquisition unit 21. The datageneration unit 16A serves as both the data generation unit 16 and thedata generation unit 26.

Note that the learning device 10A may include the data acquisition unit21 instead of the data acquisition unit 11A. In addition, the learningdevice 10A may include the data generation unit 26 instead of the datageneration unit 16A. The learning device 10A may include both of thedata acquisition unit 11 and the data acquisition unit 21. The learningdevice 10A may include both of the data generation unit 16 and the datageneration unit 26.

For the machine learning in a preparation stage of inspection, thelearning device 10A acquires and learns entirety data as a normal sampleof the inspection target, thereby generating and storing the learnedmodel 31. For an operation stage of inspection following the machinelearning, the learning device 10A acquires the entirety data of theinspection target, and determines an anomaly in the inspection targetusing the learned model 31 generated and stored by the machine learning.

When the learning device 10A executes the machine learning of the normaldata in the preparation stage of inspection, the data acquisition unit11A acquires the entirety data of the normal learning sample to be usedin the learning of the normal data. In the data generation unit 16A, thedata cutting unit 12A cuts off one or more divided data pieces from theentirety data, and the position information adding unit 13A adds cuttingposition information to each of the divided data pieces. The datageneration unit 16A sends the cutting position information added by theposition information adding unit 13A and the divided data piecesgenerated by the data cutting unit 12A to the model generation unit 14in association with each other. The model generation unit 14 generatesthe learned model 31 using the correspondence information in which thecutting position information and the divided data are associated witheach other. The learned model 31 is stored in the learned model storageunit 15A.

When the learning device 10A determines an anomaly in the operationstage of inspection, the data acquisition unit 11A acquires the entiretydata of the inspection target data. In the data generation unit 16A, thedata cutting unit 12A cuts off one or more divided data pieces from theentirety data, and the position information adding unit 13A adds cuttingposition information to each of the divided data pieces. The datageneration unit 16A sends the cutting position information added by theposition information adding unit 13A and the divided data piecesgenerated by the data cutting unit 12A to the inference unit 24 inassociation with each other. The inference unit 24 executes theinference of the normal data using the correspondence information inwhich the cutting position information and the divided data piece areassociated with each other and the learned model 31 stored in thelearned model storage unit 15A. The inference unit 24 executes theinference of the normal data by processing similar to the processingdescribed with reference to FIGS. 4 and 5 .

In the learning device 10A, the divided data in machine learning isfirst divided data, and the cutting position information in learning isfirst identification information. In addition, in the learning device10A, the correspondence information in machine learning is firstcorrespondence information, and the normal data in machine learning isfirst normal data.

In addition, in the learning device 10A, the divided data in inferenceis second divided data, and the cutting position information ininference is second identification information. In addition, in thelearning device 10A, the correspondence information in inference issecond correspondence information, and the normal data in inference issecond normal data.

Note that the learning device 10 may be built into the inference device20, or otherwise the inference device 20 may be built into the learningdevice 10. In addition, at least one of the learning device 10 and theinference device 20 may be built into an imaging device that capturesimage data of the entirety data. In addition, the learning device 10 andthe inference device 20 may exist on a cloud server.

FIG. 7 is a diagram illustrating a hardware configuration of thelearning device illustrated in FIG. 6 . The learning device 10A can beimplemented by an input device 103, a processor 101, a memory 102, andan output device 104. The processor 101 is exemplified by a centralprocessing unit (CPU, also referred to as a central processing device, aprocessing device, a computation device, a microprocessor, amicrocomputer, a digital signal processor (DSP), or a graphicsprocessing unit (GPU)), or a system large scale integration (LSI). Thememory 102 is exemplified by a random access memory (RAM) or a read onlymemory (ROM).

The learning device 10A is implemented by the processor 101 reading andexecuting a computer-executable learning program stored in the memory102 for executing the operation of the learning device 10A. It can alsobe said that the learning program that is a program for executing theoperation of the learning device 10A causes a computer to execute theprocedure or method for the learning device 10A.

The learning program to be executed by the learning device 10A has amodule configuration including the data acquisition unit 11A, the datageneration unit 16A, the model generation unit 14, and the inferenceunit 24, which are loaded on a main storage device and produced on themain storage device.

The input device 103 receives entirety data and sends the entirety datato the processor 101. The memory 102 is used as a temporary memory whenthe processor 101 executes various processes. In addition, the memory102 stores entirety data, divided data, cutting position information,the learned model 31, normal data, the anomaly indication map 33, and soon. The output device 104 outputs the normal data, the anomalyindication map 33, and so on to an external device.

The learning program may be stored in a computer-readable storage mediumin an installable or executable file and provided as a computer programproduct. Alternatively, the learning program may be provided to thelearning device 10A via a network such as the Internet. Note that a partof the function of the learning device 10A may be implemented bydedicated hardware such as a dedicated circuit, and the other partthereof may be implemented by software or firmware. In addition, thelearning device 10 and the inference device 20 can also be implementedby a hardware configuration similar to that of The learning device 10A.

In a case where the learning device 10 carries out unsupervisedlearning, other publicly known methods may be used besides the deeplearning described above. For example, the learning device 10 may usenon-hierarchical clustering based on K-means.

In addition, the model generation unit 14 may learn the normal dataaccording to the entirety data generated for a plurality of products.Note that the model generation unit 14 may acquire entirety data from aplurality of products used in one and the same area, or may learn thenormal data using the entirety data collected from a plurality ofproducts in different areas. In addition, for the model generation unit14, it is possible to add a data collection device such as an imagingdevice from which entirety data is collected to the objects, or toremove some data collection device from the objects. Furthermore, alearning device that has learned normal data for a certain product maybe applied to a different learning device, and that different learningdevice may relearn normal data to update the learned model.

Here, normal data learning processing and inference processing usingcutting position information will be described. FIG. 6 is a diagram forexplaining normal data learning processing and inference processingperformed by the learning device illustrated in FIG. 6 . Note thatdescription is given for a case where the learning device 10A learnsnormal data, but the learning of the normal data may be performed by thelearning device 10. In addition, the inference for anomaly determinationmay be performed by the inference device 20.

First, learning processing for the machine learning in a preparationstage of inspection will be described, in which entirety data isacquired and learned as the normal sample of the inspection target andthe learned model 31 is generated.

The data acquisition unit 11A of the learning device 10A acquires anentirety image 51 that is a full-size image for learning as the entiretydata of the normal learning sample of the inspection target to be usedin machine learning. The data cutting unit 12A divides the entiretyimage 51 into specific sizes, and cuts off divided data pieces D1 to Dn(n is a natural number) from the entirety image 51.

In addition, the position information adding unit 13A adds cuttingposition information pieces P1 to Pn to the divided data pieces D1 to Dnthat are a group of images, respectively. The position informationadding unit 13A adds cutting position information Pm to divided data Dm(m is a natural number of one to n). For example, the data cutting unit12A divides the entirety image 51 whose side consists of 1024 pixelsinto the dvided data pieces D1 to Dn each of which has a side consistingof 64 pixels. In this case, adjacent divided data pieces may overlapwith each other in their partial areas.

The model generation unit 14 inputs, to the learned model 31,correspondence information in which the divided data Dm and the cuttingposition information Pm are associated with each other. In this manner,the model generation unit 14 cuts the entirety image 51 that is afull-size image for learning into specific sizes, and inputs a set ofthe divided data Dm that is a group of cutaway images and the cuttingposition information Pm simultaneously to the learned model 31. In thisexample, machine learning is performed through deep learning such thatfeatures of the image of the normal divided data Dm are extracted in theintermediate layer of the neural network in association with a set ofthe divided data Din and the cutting position information Pm, and animage of normal data appropriate as normal data having the extractedfeatures is outputted. Consequently, the learned model 31 is subjectedto machine learned such that correspondence information in which normaldata that is a learning result and the cutting position information Pmcorresponding to this normal data are associated with each other isoutputted.

In the machine learning, a plurality of learning samples are usuallyprepared, and machine learning is performed based on multiple entiretydata sets obtained from the plurality of learning samples. Consequently,machine learning is executed such that normal data appropriate as normaldata in which variations in the range determined to be normal areconsidered can be outputted. The variations can include, for example,variations in placement of components attached to the inspection target,and variations an environment such as brightness when capturing an imageof the inspection target with a camera or the like to acquire an image.

The model generation unit 14 sequentially inputs sets of the divideddata Dm and the cutting position information Pm to one learned model 31thereby to sequentially perform machine learning of the normal datacorresponding to the divided data pieces D1 to Dn. The learned model 31is subjected to machine learning such that correspondence information inwhich normal data as a learning result and the cutting positioninformation Pm corresponding to the normal data are associated with eachother can be outputted for each divided data piece.

When inputting, to the learned model 31, one correspondence informationset in which the divided data Dm and the cutting position information Pmare combined, the model generation unit 14 divides the correspondenceinformation set into information pieces each of which is per one pixelincluded in the correspondence information set.

The model generation unit 14 inputs the divisional information piece ofeach pixel to the input layer of the learned model 31. That is, themodel generation unit 14 performs machine learning such that the divideddata piece Dm into pixels, input the data of each pixel to the learnedmodel 31, and output normal data from the learned model 31 for eachpixel.

For example, in a case where the divided data piece Dm has one side of64 pixels, the model generation unit 14 inputs the data of 64×64 pixelsto the learned model 31, and causes the learned model 31 to output thedata of 64×64 normal pixels.

The learned model 31 generated by the model generation unit 14 cangenerate one image of normal data corresponding to one image of divideddata on the basis of the data of pixels outputted by the learned model31. The image generated as the normal data is divided data Rm. Thedivided data Pm corresponds to the divided data Dm and is associatedwith the cutting position information Pm. The learned model 31 outputscorrespondence information in which the divided data Rm that is normaldata is associated with the cutting position information Pm. Thegenerated learned model 31 is stored in the learned model storage unit15A.

Note that the learned model 31 may associate an input layer and an oflayer with each pixel so that the received pixel and the outputted pixelare at the same position, and generate one piece of divided data Rm onthe basis of the positions of the input layer and the output layer ofeach pixel. In addition, the cutting position information correspondingto the divided data Rm as the normal data outputted from the learnedmodel 31 when the cutting position information Pm corresponding to thedivided data Dm is inpuLted to the learned model 31 may be used withreference to the cutting position information Pm inputted to the learnedmodel 31. Therefore, the learned model 31 may output only the divideddata Rm as normal data without outputting the cutting positioninformation.

Next, inference processing in an operation stage of inspection followingthe machine learning will be described, in which the entirety data ofthe inspection target is acquired and an anomaly in the inspectiontarget is determined using the learned model 31 generated and stored inthe machine learning.

The data acquisition unit 11A of the learning device 10A acqures theentirety image 51 that is a full-size image of the inspection target asthe entirety data to be used for inspection. The data cutting unit 12Adivides the entirety image 51 into specific sizes, and cuts off thedivided data pieces D1 to Dn from the entirety image 51, as in thelearning processing in the preparation stage.

In addition, the position information adding unit 13A adds the cuttingposition information pieces P1 to Pn respectively to the divided datapieces D1 to Dn that are a group of images, as in the learningprocessing. The position information adding unit 13A adds the cuttingposition information Pm to the divided data Dm. For example, the datacutting unit 12A divides the entirety image 51 having one side of 1024pixels into the divided data pieces D1 to Dn having one side of 64pixels. Adjacent divided data pieces may overlap with each other intheir partial areas.

The inference unit 24 acquires in advance the learned model 31 subjectedto machine learning and stored in the learned model storage unit 15A inthe learning processing in the preparation stage, and inputscorrespondence information in which the cutting position information Pmand the divided data Dm are associated with each other, to the learnedmodel 31. In this manner, the inference unit 24 cuts the entirety image51 that is a full-size image for learning into specific sizes, andinputs a set of the divided data Dm that is a group of the cutawayimages and the cutting position information Pm simultaneously to thelearned model 31. Consequently, the learned model 31 outputscorrespondence information in which normal data as a learning result andthe cutting position information Pm corresponding to the normal data areassociated with each other. Note that the cutting position informationcorresponding to the divided data Rm as the normal data outputted fromthe learned model 31 may be identified with reference to the cuttingposition information Pm inputted to the learned model 31, and so thelearned model 31 may output only the divided data Rm as normal datawithout outputting the cutting position information.

The inference unit 24 sequentially inputs sets of the divided data Dmand the cutting position information Pm of the inspection target to onelearned model 31, thereby to sequentially infer the normal datacorresponding to the divided data pieces D1 to Dn. The learned model 31outputs, for each piece of divided data, correspondence information inwhich normal data as an inference result and the cutting positioninformation Pm corresponding to the normal data are associated with eachother. In FIG. 8 , the normal data outputted by the learned model 31 isrepresented by a group of images of divided data pieces R1 to Rn.

When inputting, to the learned model 31, one correspondence informationset in which the divided data Dm and the cutting position information Pmof the inspection target are combined, the inference unit 24 divides thecorrespondence information into information pieces each of which is perpixel included in the correspondence information.

The inference unit 24 inputs the divisional information piece of eachpixel to the input layer of the learned model 31. That is, the inferenceunit 24 divides each piece of divided data Dm into pixels, inputs thedata of each pixel to the learned model 31, and causes the learned model31 to output normal data for each pixel.

For example, in a case where the divided data piece Dm has one side of64 pixels, the inference unit 24 inputs the data of 64×64 pixels to thelearned model 31, and causes the learned model 31 to output the data of64×64 normal pixels.

The inference unit 24 generates one image of normal data correspondingto one image of divided data on the basis of the data of pixelsoutputted from the learned model 31. The image generated as the normaldata is the divided data Rm. The divided data Rm corresponds to thedivided data Dm and is associated with the cutting position informationPm. The learned model 31 outputs correspondence information in which thedivided data Rm that is normal data is associated with the cuttingposition information Pm.

The inference unit 24 compares the divided data Rm with the divided dataDm, and generates divided data Tm representing a difference between Rmand Dm. That is, the inference unit 24 compares the divided data Dminputted to the learned model 31 with the divided data Rm outputted fromthe learned model 31, and generates the divided data Tm that is acomparison result. That is, the inference unit 24 generates divided datapieces T1 to Tn corresponding to the divided data pieces D1 to Dn. Thedivided data pieces D1 to Dn correspond to the divided data pieces R1 toRn, respectively, and the divided data pieces R1 to Rn correspond to thedivided data pieces T1 to Tn, respectively.

The divided data Tm is data representing a difference between thedivided data Rm that is normal data and the divided data Dm that isinput data. Therefore, the divided data Tm data representing an abnormalportion.

The inference unit 24 generates the anomaly indication map 33 using thedivided data pieces T1 to Tn that are a group of images. Specifically,the inference unit 24 generates the anomaly indication map 33corresponding to the entirety image 51 by recombining the divided datapieces T1 to Tn outputted from the learned model 31 and superimposingthe resultant of the recombination on the entirety image 51.

In this manner, the inference unit 24 determines an anomaly in thedivided data Rm by comparing the divided data Rm, i.e., normal datainferred by the learned model 31 based on the divided data Dm and thecutting position information Pm, with the divided data Dm received fromthe data generation unit 16A, and identifies a portion having anabnormal state an the divided data Rm. Then, the inference unit 24generates, based on the thus-obtained identification result, the anomalyindication map 33 in which the portion having an abnormal state in theentirety image 51 is specified.

Now the description is given for a difference between the anomalyindication map 33 generated based on the learned model 31 subjected tomachine learning using the divided data Dm and the cutting positioninformation pieces P1 to Pn and the anomaly indication map generatedbased on the learned model 31 subjected to machine learning using thedivided data Dm without using the cutting position information pieces P1to Pn. FIG. 9 is a diagram for explaining the difference between ananomaly indication map generated by the inference device according tothe first embodiment using the cutting position information and anotheranomaly indication map generated without using the cutting positioninformation.

Here, a case where the anomaly indication maps are generated for aninspection target image 70 including abnormal portions 35 and 36 will bedescribed. The inspection target image 70 is a full-size image forlearning, which is obtained by capturing an image of a specific product.The abnormal portion 35 is a portion where arrangement of components hasposition aberration. The abnormal portion 36 is a portion that lacks alabel seal which should be present intrinscally.

The anomaly indication map 33X is an anomaly indication map generatedfor the inspection target image 70 using the learned model 31 subjectedto machine learning without using the cutting position informationcorresponding to the divided data Dm. That is, the anomaly indicationmap 33X is an inspection result image obtained in a case where themachine learning is performed without using identification informationfor identifying regions of the divided data Dm in the full-size learningtarget data.

The abnormal portion 36 included in the inspection target image 70 isdivided into pieces of divided data Dm as with the other portions. Inthis case, because each piece of the divided data Dm is small, thedivided data Dm in the abnormal portion 36 may be substantially the sameas the normal divided data Dm. That is, there may be a case where thedivided data Dm in the abnormal portion 36 is similar to the divideddata Dm in some place other than the abnormal portions 35 and 36. Inother words, when the divided data Dm is cut, there may be a normalimage portion similar to an abnormal portion that lacks a label someplace other than the abnormal portions 35 and 36. In this case, thelearned model 31 subjected to machine learning without usingidentification information corresponding to the divided data Dm cannotdistinguish between the divided data Dm in the abnormal portion 36 and asimilar normal image portion, and outputs normal data that is thelearning result of the similar normal image portion. Therefore, thedivided data Dm in the abnormal portion 36 is identified as the normaldivided data Dm by being compared with the normal data of the similarnormal image portion outputted from the learned model 31. Thus, theanomaly indication map 33X represents the abnormal portion 35 as anabnormal portion, but erroneously represents the abnormal portion 36 asa normal portion.

The anomaly indication map 33A is an anomaly indication map generatedfor the inspection target image 70 using the learned model 31 subjectedto machine learning using the divided data Dm and the cutting positioninformation pieces P1 to Pn. That is, the anomaly indication map 33A isan inspection result image obtained in a case where the machine learningis performed using the divided data Dm and identification informationfor identifying regions of the divided data Dm in the full-size learningtarget data.

In the case where the learning device 10A generates the learned model 31trough machine learning using the cutting position information pieces P1to Pn, the divided data Dm have different pieces of cutting positioninformation Pm even if the divided data Dm of the abnormal portion 36 issimilar to the divided data Dm in some other normal portion. Therefore,the learning device 10A can output the learning result withdistinguishing between the divided data Dm of the abnormal portion 36and the divided data Dm in the other normal portion. As a result, theinference unit 24 of the learning device 10A can accurately determinewhether or not there is an anomaly in each piece of divided data Dm ofthe inspecton target based on the divided data Dm of the inspectiontarget and the cutting position information pieces P1 to Pn.

Although the first embodiment has been described for a case where thelearning target and the inspection target are two-dimensional imagedata, the learning target and the inspection target may beone-dimensional data such as a data string in which data values of atime-series waveform are lined up at regular time intervals, or may bemulti-dimensional data in which multiple pieces of data are combined.Data combined as multi-dimensional data may be measurement data such aselectric current value data, or may be image data. One-dimensional datais exemplified by an electric current waveform that is a time-serieswaveform obtained by measuring the value of an electric current flowingthrough the product during the operation of the product. If theoperation of the product includes a plurality of operation stages, thereis a feature of the current waveform for each time region of each of theoperation stages have different current waveform features. In this case,the learning device 10 generates divided data obtained by dividing thecurrent waveform into a plurality of time regions, and performs machinelearning of a normal divided current waveform having normal features foreach time region with use of the correspondence information in which thedivided current waveform which is the divided data of each time regionand region identification information foridentifying the cdvided data ofeach time region are associated with each other to generate the learnedmodel 31. In addition, the inference device 20 acquires the currentwaveform of the product that is an inspection target, divides thecurrent waveform into divided data pieces of a plurality of timeregions, infers the normal data of the current for each time regionbased on the learned model 31 using correspondence information in whichthe divided data and the region identification information foridentifying each time region are associated with each other, andcompares the divided data with the normal data, so as to determine ananomaly in the current waveform of the inspection target.

The learning device 10 and the inference device 20 are applied to, forexample, inspection of products in the manufacturing industry (e.g.,appearance inspection using images). Because the learning device 10 andthe inference device 20 divide the entirety data into the divided datapieces D1 to Dn and infer the normal data using the cutting positioninformation pieces P1 to Pn, they can efficiently learn the normal dataat reduced learning cost. The learning cost as said herein includes acalculation time, the number of computers, the number of learningsamples, and the like.

In addition, because the learning device 10 and the inference device 20can reduce the learning cost, it is possible to perform the machinelearning of the inspection target in a short time even when there aremore than several hundreds or several thousands types of products oreven when there are a large number of inspection processes in a factorymanufacturing personal computer (PC) substrates, factory automation (FA)devices, or the like. In addition, because the learning device 10 andthe inference device 20 can reduce the learning cost, it is possible toaccurately perform the machine learning of the inspection target in ashort time even when the machine learning is executed for each of alarge number of models or for each of a large number of processes.

As described above, in the first embodiment, the learning device 10 andthe inference device 20 divide the entirety data to generate multiplepieces of divided data Dm, and add the cutting position information Pmto each piece of divided data Dm. The learning device 10 and theinference device 20 generate the learned model 31 for inferring thenormal data from the correspondence information that is a set of thedivided data Dm and the cutting position information Pm. Consequently,the learning device 10 and the inference device 20 can accuratelyperform machine learning of the normal state of the inspection target ina short time.

In addition, the inference device 20 compares the normal data with thedivided data Dm based on the cutting position information Pm to identifyan abnormal portion in the divided data Dm, and identifies the portionhaving an abnormal state in the entirety data of the inference targetbased on the identification result. Consequently, the inference device20 can identify the portion having an abnormal state in the entiretydata in a short time.

In the first embodiment, as identification information for identifyingthe region of each piece of divided data in the entirety data, cuttingposition information that is information of the position where eachpiece of divided data is cut from the entirety data is used. However,the identification information may be anything that enables theidentification of multiple pieces of divided data cut from the entiretydata. For example, if 256 pieces of divided data are generated from oneentirety data set, integers of 0 to 255 may be assigned thereto. In thatcase, the cutaway pieces of divided data only need to correspondone-to-one to the assigned pieces of identification information, and thearrangement order of the pieces of divided data in the entirety data maybe different from the order of the integers assigned as theidentification information.

Second Embodiment

Next, the second embodiment will be described with reference to FIGS. 10to 12 . In the second embodiment, the learning device 10 combinesinformation representing cutting position information as an image(cutting position image to be described later) with divided data, andexecutes machine learning using the information obtained by thecombination.

FIG. 10 is a diagram for explaining a cutting position image set individed data of three channels of RGB colors by the learning deviceaccording to the second embodiment. In a case where a learning image isa full-size image 75 of three channels of RGB (red, green, and blue)colors, the full-size image 75 includes a red image 75R, a green image75G, and a blue image 75B as images for the three channels.

The data generation unit 16 of the learning device 10 cuts off a divideddata piece 77 from the image data of the full-size image 75, andcombines information representing the cutting position information ofthe divided data piece 77 as an image with the divided data piece 77.The divided data piece 77 cut from a specific region of the full-sizeimage 75 by the data generation unit 16 includes the divided data piecesfor the three channels. That is, the divided data piece 77 cut from aspecific region of the full-size image 75 by the data generation unit 16includes a red divided data piece 77R, a green divided data piece 77G,and a blue divided data piece 77B.

Note that in FIG. 10 , the images 75R, 75G, and 75B are separatelyillustrated, but the data generation unit 16 does not necessarily needto decompose the full-size image 75 into the images 75R, 75G, and 75B.Similarly, in FIG. 10 , the divided data pieces 77R, 77G, and 77B areseparately illustrated, but the data generation unit 16 does notnecessarily need to decompose the divided data piece 77 into the divideddata pieces 77R, 77G, and 77B.

The data generation unit 16 acquires a cutting position image 76Pcorresponding to the divided data piece 77 including the information ofthe divided data pieces 77R, 77G, and 77B. That is, the data generationunit 16 acquires the cutting position image 76P representing theposition of the divided data piece 77 with respect to the full-sizeimage 75. The cutting position image 76P is data obtained by imaging thecutting position information of the divided data piece 77. An example ofthe cutting position image 76P is an image in which the cutting positionof the divided data piece 77 is expressed in white and the other part isexpressed in black.

Furthermore, the data generation unit 16 resizes the cutting positionimage 76P so that an aspect ratio (ratio of width to height) and animage size of the cutting position image 76P become equal to an aspectratio and an image size of the divided data piece 77. Consequently, thecutting position image 76P for the divided data piece 77 becomes acutting position image 78P having the same aspect ratio and image sizeas the divided data piece 77.

The data generation unit 16 combines the cutting position image 78P withthe divided data piece 77 including the divided data pieces 77R, 77G,and 77B. That is, the data generation unit 16 adds the cutting positionimage 78P to the divided data piece 77 as data for a fourth channel.Consequently, the divided data piece 77 becomes a divided data piece 79that is a four-channel image for red, green, blue, and position. Themodel generation unit 14 inputs the four-channel image, namely thedivided data piece 79, to the learned model 31. As a result, thelearning device 10 can easily perform machine learning by just changingthe number of channels without significantly changing the structure ofthe learned model 31.

The method of combining the cutting position image 78P with the divideddata piece 77 can also be applied to a case where the mill-size imagehas one channel for grayscale. In addition, the method of combining thecutting position image 78P with the divided data piece 77 can also beapplied without dependence on any channel to a case where the full-sizeimage is an RGB-depth image including depth (distance) information inaddition to RGB.

Note that the inference device 20 may execute the processing describedwith reference to FIG. 10 in the operation stage of inspection. In thiscase, the inference device 20 combines the cutting position image 78Pwith the divided data piece 77, and executes anomaly determination usingthe information obtained by the combination, namely the divided datapiece 79.

FIG. 11 is a diagram for explaining a cutting position image set individed data of one channel for grayscale by the learning deviceaccording to the second embodiment. In a case where the image data ofthe learning target is a full-size image 85 of one channel forqrayscale, the full-size image 85 includes a grayscale image for onechannel.

The data generation unit 16 of the learning device 10 cuts off a divideddata piece 87 from the image data of the full size image 85, andcombines information representing the cutting position information ofthe divided data piece 87 as an image with the divided data piece 87.The divided data piece 87 cut from a specific region of the full-sizeimage 85 by the data generation unit 16 includes the divided data piecefor one channel.

The data generation unit 16 acquires a cutting position image 86Pcorresponding to the divided data piece 87. That is, the data generationunit 16 acquires the cutting position image 86P representing theposition of the divided data piece 87 with respect to the full-sizeimage 85. The cutting position image 86P is data obtained by imaging thecutting position information of the divided data piece 87. An example ofthe cutting position image 86P is an image in which the cutting positionof the divided data piece 87 is expressed in white and the other part isexpressed in black.

Furthermore, the data generation unit 16 resizes the cutting positionimage 86P so that the aspect ratio and the image size of the cuttingposition image 86P become the same as the aspect ratio and the imagesize of the divided data piece 87. Consequently, the cutting positionimage 86P for the divided data piece 87 becomes a cutting position image88P having the same aspect ratio and image size as the divided datapiece 87. The cutting position image 86P is information similar to thecutting position image 76P, and the cutting position image 88P isinformation similar to the cutting position image 78P.

The data generation unit 16 combines the cutting position image 88P withthe divided data piece 87. That is, the data generation unit 16 adds thecutting position image 88P to the divided data piece 87 as data for thesecond channel. As a result, the divided data piece 87 becomes a divideddata piece 89 that is a two-channel image for grayscale and position.The model generation unit 14 inputs the two image, namely the divideddata piece 89, to the learned model 31. Consequently, the learningdevice 10 can easily perform machine learning with just changing thenumber of channels without significantly changing the structure of thelearned model 31.

Note that the inference device 20 may execute the processing describedwith reference to FIG. 11 in the operation stage of inspection. In thiscase, the inference device 20 combines the cutting position image 88Pwith the divided data piece 87, and executes anomaly determination usingthe information obtained by the combination, namely the divided datapiece 89.

Note that the learning device 10 may use a data array image by which adivided-data cutting region can be identified, instead of the cuttingposition image 88P set in the divided data of one channel for grayscale.

FIG. 12 is a diagram for explaining a data array image by which adivided-data cutting region can be identified and which is set in thedivided data of one channel for grayscale by the learning deviceaccording to the second embodiment. The data generation unit 16 of thelearning device 10 cuts off the divided data piece 87 from the full-sizeimage 85 in substantially the same processing as the processingdescribed with reference to FIG. 11 . In addition, the data generationunit 16 sets, in the divided data piece 87, a data array image by whichthe cutting region of the divided data piece 87 in the full-size image85 can be identified. The data array image is a two-dimensional code.

Hereinafter, the data array image having such identifiability referredto as an identification data image 88Q. An example of the identificationdata image 88Q is a quick response (QR) code (registered trademark). Theidentification data image 88Q is information for identifying the regionof the divided data piece 87 in the full-size image 85, which is usedhere instead of the cutting position images 78P and 88P.

The data generation unit 16 combines the identification data image 88Qwith the divided data piece 87. That is, the data generation unit 16adds the identification data image 88Q to the divided data piece 87 asdata for the second channel. As a result, the divided data piece 87becomes a divided data piece 90 that is a two-channel image forgrayscale and position. The model generation unit 14 inputs thetwo-channel image, namely the divided data piece 90, to the learnedmodel 31. Consequently, the learning device 10 can easily performmachine learning with just changing the number of channels withoutsignificantly changing the structure of the learned model 31.

The data generation unit 16 may apply the identification data image 88Qto the full-size image 75 of the three channels for RGB colors. Inaddition, the inference device 20 may execute the processing describedwith reference to FIG. 12 . In this case, the inference device 20combines the identification data image 88Q with the divided data piece87, and executes machine learning using the information obtained by thecombination, namely the divided data piece 90. In addition, theinference device 20 may execute the inference of normal data using thedivided data piece 90.

In addition, in the data generation units 16 and 26, an image obtainedby inverting white and black of the cutting position images 76P, 78P,86P, and 88P may be used as their cutting position images. Moreover, anycolors may be used for the cutting position images 76P, 78P, 86P, and88P, not only white and black. Also in this case, the data generationunits 16 and 26 treat the cutting position images 76P, 78P, 86P, and 88Pso that a region of the full-size image occupied by the divided data isexpressed in a first color, and another region of the full-size imagethat is not occupied by the divided data is expressed in a second color.

In addition, the data generation units 16 and 26 may use any informationas identification information of the divided data pieces 77 and 87 aslong as it is information that is intended to identify the regions ofthe divided data pieces 77 and 87 and corresponds one-to-one to thedivided data pieces 77 and 87. For example, the data generation units 16and 26 may use a one-dimensional code such as a one-dimensional barcode,as information for identifying the regions of the divided data pieces 77and 87.

Now the description is given for a difference between the calculationtime in the case where the learning device 10 or the inference device 20divides the full-size image 75 to execute machine learning as describedwith reference to FIG. 10 and the calculation time in the case wheremachine learning is executed without dividing the full-size image 75.

FIG. 13 is a table for explaining the difference between the calculationtime in the case where the inference device according to the secondembodiment divides a full-size image to execute machine learning and thecalculation time in the case where the machine learning is executedwithout dividing the full-size image. Here, the difference incalculation time will be described on the premise that the machinelearning is performed by means of deep learning in which neural networksare stacked in multiple layers.

“Full-size learning” is machine learning performed using the full-sizeimage 75 as is, and “cutting learning” is machine learning performedafter dividing the full-size image 75 into pieces of divided data. Thisexample is based on the assumption that an image has each side of 1024pixels in the case of “full-size learning”, and an image each side of 64pixels in the case of “cutting learning”. That is, the full-size image75 has 1024 pixels constituting one side thereof, and an image of thedivided data piece has 64 pixels constituting one side thereof.

In the case of “full-size learning”, the number of channels is threechannels for R, G, and B, and the total number of pixels is1024×1024×3=3145728. In the case of “cutting learning”, the number ofchannels is four channels for R, G, B, and position, and the totalnumber of pixels is 64×64×4=16384.

The above-described “full-size learning” was executed on 60 full-sizeimages 75, and the learning time thereby was about 720 minutes. On theother hand, in the case of the above-described “cutting learning”, thefull-size image 75 whose number of pixels is 1024×1024 is divided into(1024×1024)/(64×64)=256 pieces of divided data. Therefore, in the caseof “cutting learning”, 60 full-size images 75 are divided into256×60=15360 pieces of divided data.

The learning device 10 or the inference device 20 executed the “cuttinglearning” on the 15360 pieces of divided data, and the learning time thewas about 40 minutes.

As described above, in the “full-size learning”, the total number ofpixels is 3145728, which corresponds to the number of input lavers ofthe learned model used in the “full-size learning”. On the other hand,in the “cutting learning”, the total number of pixels is 16384, which isless than 1/100 of that in the “full-size learning”. Therefore, in the“cutting learning”, the number of input layers of the learned model isalso less than 1/100 of that in the “full-size learning”, and the numberof connections having weights among the input, intermediate, and outputlayers can also be greatly reduced. As a result, the memory capacityrequired for machine learning can be significantly reduced as comparedwith the case of the “full-size learning”. In the case of “cuttinglearning”, the total number of images was larger than in the case of“full-size learning”, but the learning time could be made about 1/20shorter than in the case of “full-size learning”.

As described above, according to the second embodiment, the learningdevice 10 combines the cutting position image 78P with the divided dataand executes machine learning using the divided data obtained by thecombination. Therefore, the learning device 10 can easily performmachine learning with just changing the number of channels withoutsignificantly changing the structure of the learned model 31.

Third Embodiment

Next, the third embodiment of the present invention will be describedwith reference to FIG. 14 . In the third embodiment, the inferencedevice 20 directly inputs the cutting position information of thedivided data pieces D1 to Dn to the intermediate layer of the learnedmodel.

FIG. 14 is a diagram for explaining processing in which the inferencedevice according to the third embodiment directly inputs the cuttingposition information of divided data to the intermediate layer of thelearned model. Note that the inference device 20 learns normal data inthe described case, but the learning of normal data may be performed bythe learning device 10.

In the third embodiment, the data generation unit 26 of the inferencedevice 20 directly inputs, to the intermediate layer of a learned model32, the cutting position information on the divided data Dm obtained bythe cutting from the entirety image 51 that is a full-size image. Anexample of the cutting position information of the divided data Dm isnumerical data such as coordinates of the divided data Dm. The learnedmodel 32 is a neural network similar to the learned model 31.

The data generation unit 26 inputs the divided data Dm to the inputlayer of the learned model 32 for the divided data Dm, and inputs thecutting position information of the divided data Dm to the intermediatelayer thereof. Consequently, the divided data Rm is outputted from theoutput layer of the learned model 32. The learned model 32 outputs thedivided data Rm corresponding to the divided data Dm in substantiallythe same manner as the learned model 31. Thus, the learned model 32outputs the divided data pieces R1 to Rn in response to the input of thedivided data pieces D1 to Dn.

The data generation unit 26 compares the divided data pieces Di to Dnwith the divided data pieces R1 to Rn and generates the divided datapieces T1 to Tn in processing similar to the processing described withreference to FIG. 8 . The data generation unit 26 generates an anomalyindication map 34 by recombining the divided data pieces T1 to Tn on thebasis of the divided data pieces T1 to Tn and the cutting positioninformation inputted to the intermediate layer of the learned model 32.

As described above, according to the third embodiment, the inferencedevice 20 directly inputs the cutting position information on thedivided data pieces D1 to Dn to the intermediate layer of the learnedmodel 32. Therefore, it is possible to execute machine learning withoutgenerating correspondence information.

The configurations described in the above-mentioned embodimentsillustrate just examples, each of which can be combined with otherpublicly known techniques or with the other, and can be partiallyomitted and/or modified without departing from the scope of the presentdisclosure.

REFERENCE SIGNS LIST

10, 10A learning device; 11, 11A, 21 data acquisition unit; 12, 12A, 22data cutting unit; 13, 13A, 23 position information adding unit; 14model generation unit; 15, 15A learned model storage unit; 16, 16A, 26data generation unit; 20 inference device; 24 inference unit; 31, 32learned model; 33, 33A, 33X, 34 anomaly indication map; 35, 36 abnormalportion; 51 entirety image; 70 inspection target image; 75, 85 full-sizeimage; 75B, 75G, 75R image; 76P, 78P, 86P, 88P cutting position image;77, 77B, 77G, 77R, 79, 87, 89, 90 divided data piece; 88Q identificationdata image; 101 processor; 102 memory; 103 input device; 104 outputdevice; D1 to Dn, R1 to Rn, T1 to Tn divided data piece; P1 to Pncutting position information piece.

1. A learning device comprising: a data acquisition unit to acquirelearning target data that is full-size data of a learning target; a datageneration unit to divide the learning target data to generate multiplepieces of first divided data that is divided data of the learning targetdata, and add, to each piece of the first divided data, firstidentification information for identifying a region of the first divideddata in the learning target data; and a model generation unit togenerate a learned model for determining an anomaly in the first divideddata using first correspondence information that is a set of the firstdivided data and the first identification information corresponding tothe first divided data.
 2. The learning device according to claim 1,wherein in response to receiving the first correspondence information,the learned model outputs first normal data appropriate as normal dataof the first divided data.
 3. The learning device according to claim 1,wherein the data acquisition unit acquires inference target data that isfull-size data of an inference target, the data generation unit dividesthe inference target data to generate multiple pieces of second divideddata that is divided data of the inference target data, and adds, toeach piece of the second divided data, second identification informationfor identifying a region of the second divided data in the learningtarget data, and the learning device further includes an inference unitto determine an anomaly in the inference target data by inputting, tothe learned model, second correspondence information that is a set ofthe second divided data and the second identification informationcorresponding to the second divided data.
 4. The learning deviceaccording to claim 3, wherein in response to receiving the secondcorrespondence information, the learned model outputs second normal dataappropriate as normal data of the second divided data, and the inferenceunit determines an anomaly in the second divided data by comparing thesecond normal data with the second divided data, and determines ananomaly in the inference target data based on a determination result. 5.The learning device according to claim 4, wherein the inference unitidentifies a portion having an abnormal state in the second divided databy comparing the second normal data with the second divided data, andidentifies a portion having an abnormal state in the inference targetdata based on an identification result.
 6. The learning device accordingto claim 1, wherein the learning target data is image data, and the datageneration unit is adapted to generate the learned model with imagingthe first identification information and resizing the firstidentification information imaged to have an image size equal to animage size of the first divided data, and generating the firstcorrespondence information using the first identification informationobtained by the imaging and resizing, to generate the learned modelusing the first correspondence information generated.
 7. The learningdevice according to claim 3, wherein the learning target data and theinference target data are image data, and the data generation unit isadapted to: generate the learned model with imaging the firstidentification information and resizing the first identificationinformation imaged to have an image size equal to an image size of thefirst divided data, and generating the first correspondence informationusing the first identification information obtained by the imaging andresizing, to generate the learned model using the first correspondenceinformation generated; and determine an anomaly in the inference targetdata with imaging the second identification information and resizing thesecond identification information imaged to have an image size equal toan image size of the second divided data, and generating the secondcorrespondence information using the second identification informationobtained by the imaging and resizing, to input the second correspondenceinformation generated to the learned model.
 8. The learning deviceaccording to claim 7, wherein the first identification information isinformation in which a region of the learning target data occupied bythe first divided data is expressed in a first color, and a region ofthe learning target data that is not occupied by the first divided datais expressed in a second color, and the second identificationinformation is information in which a region of the inference targetdata occupied by the second divided data is expressed in the firstcolor, and a region of the inference target data that is not occupied bythe second divided data is expressed in the second color.
 9. Thelearning device according to claim 8, wherein when the first color iswhite, the second color is black, and when the first color is black, thesecond color is white.
 10. The learning device according to claim 7,wherein the first identification information and the secondidentification information are two-dimensional codes.
 11. The learningdevice according to claim 3, wherein the learned model is a neuralnetwork, the model generation unit inputs the first identificationinformation to an intermediate layer of the neural network to generatethe learned model, and the inference unit inputs the secondidentification information to the intermediate layer of the neuralnetwork to determine an anomaly in the inference target data.
 12. Thelearning device according to claim 3, wherein the learning target dataand the inference target data are one-dimensional data.
 13. A learningmethod comprising: a data acquisition step of acquiring learning targetdata that is full-size data of a learning target; a data generation stepof dividing the learning target data to generate multiple pieces offirst divided data that is divided data of the learning target data, andadding, to each piece of the first divided data, first identificationinformation for identifying a region of the first divided data in thelearning target data; and a model generation step of generating alearned model for determining an anomaly in the first divided data usingfirst correspondence information that is a set of the first divided dataand the first identification information corresponding to the firstdivided data.
 14. An inference device comprising: a data acquisitionunit to acquire inference target data that is full-size data of aninference target; a data generation unit to divide the inference targetdata to generate multiple pieces of divided data of the inference targetdata, and add, to each piece of the divided data, identificationinformation for identifying a region of the divided data in theinference target data; and an inference unit to determine an anomaly inthe inference target data using correspondence information that is a setof the divided data and the identification information corresponding tothe divided data and a learned model for determining an anomaly in thedivided data using the correspondence information.
 15. A storage mediumin which a program is stored, the program comprising: a data acquisitionstep of acquiring learning target data that is full-size data of alearning target; a data generation step of dividing the learning targetdata to generate multiple pieces of first divided data that is divideddata of the learning target data, and adding, to each piece of the firstdivided data, first identification information for identifying a regionof the first divided data in the learning target data; and a modelgeneration step of generating a learned model for determining an anomalyin the first divided data using first correspondence information that isa set of the first divided data and the first identification informationcorresponding to the first divided data.